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A 2005 Toyota Prius, which was in an accident, is seen ata police station in Harrison, New York, Wednesday, 
peeled 2010. The driver of the Toyota Prius told police that the car accelerated on its own, then lurched down a 
driveway, across a road and into 4 stone wall. (AP Photo/Seth Wenig) AP PHOTO/SETH WENIG 
Unintended acceleration in Toyota vehicles may have been involved in the deaths 
of 89 people over the past decade, upgrading the number of deaths possibly linked 
to the massive recalls, the government said Tuesday. 


The National Highway Traffic Safety Administration said that from 2000 to mid- 
May, it had received more than 6,200 complaints involving sudden acceleration in 
Toyota vehicles. The reports include 89 deaths and 57 injuries over the same 
period. Previously, 52 deaths had been suspected of being connected to the 
problem. 


See: https://users.ece.cmu.edu/~koopman/toyota/index.html 


It's All Your Fault: The DOT Renders Its 
Verdict on Toyota's Unintended- 
Acceleration Scare 


The final word on the Toyota unintended-acceleration mess. 
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™ From the June 2011 issue of Car and Driver 
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Tesla Blames Driver In Fatal Model X Autopilot yak 
Crash As Family Considers Legal Action — *»/vmscarm 
pra foe hel to: TESLA v 2A GK 325 





Huang fatality; 
crash into 


concrete median. 
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Tesla car suddenly and 
unintentionally accelerated 
into driver's house, lawsuit 
Says 


Ji Chang Son says that Tesla's Model X vehicles are plagued by the 
phenomenon 
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& Amy Martyn 
Reporter 








Feds blame driver error for 
16,000 annual unintended 
acceleration cases 


Accident victims don't buy the argument but few can document any 
other cause 
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4&& James R. Hood 
“Founder and Editor 





Peggy's Jeep 3/7 


Can Humans Safely Supervise Autonomy? 


Man reportedly caught sleeping Google's Waymo Self-Driving Car Crashed After 





behind the wheel of a self- Driver Dozed Off Back in June 
driving Tesla https://goo.gl/ZFCYzD Qa Justin T. Westbrook 
10/04/18 10:28am + Filed to: WAYMO v 


Sarah Whitten | @sarahwhit10 
Published 11:38 AM ET Wed, 25 May 2016 | Updated 9:46 AM ET Thu, 26 May 2016 
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A Waymo self-driving car sent a 
motorcyclist to the hospital — but 


== the human driver was at fault 
BUSINESS 
Graham Rapier Nov. 6, 2018, 4:20 PM | N S | D E Fe 
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https://goo.gl/VTFW9d 


Photo: Waymo 


waymo 





https://goo.gl/kgRq71 
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The 94% Human Error False Narrative Dart 





m= Where did "94%" come from? 
e “The critical reason was 
assigned to drivers in an estimated 
2,046,000 crashes that comprise 
94 percent of the NMVCCS crashes 
at the national level. 


However, in none of these cases was 


the assignment intended to blame 


the driver for causing the crash.” 
[DOT HS 812 115] 


m Looking a little deeper: 


e 74% of driver errors were “recognition” or “decision” errors 


“Q4o,,” 





pl EY 


Benefits of Automation 


SAFETY 


The safety benefits of automated vehicles are paramount. Automated 
vehicles’ potential to save lives and reduce injuries is rooted in one 
critical and tragic fact: 94 percent Of Serious Crashes are due to human 
fron Automated vehicles have the potential to remove human error 
from the crash equation, which will help protect drivers and 
passengers, as well as bicyclists and pedestrians. When you consider 
more than 35,092 people died in motor vehicle-related crashes in the 
U.S. in 2015, you begin to grasp the lifesaving benefits of driver 
assistance technologies. 


https://www.nhtsa.gov/technology- 
innovation/automated-vehicles-safety 


e And software driver must handle the 6% of no-driver-involvement crash causes 


— Tires, brakes, drivetrain failures 
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Humans Are Amazing Fault Mitigators ie", 


= Other side of the "94%" coin — people prevent crashes too 


= Toyota uncommanded acceleration — most saved by human 
e 89 deaths, 57 injuries as of May 2010 
@ 6,200+ NHTSA complaints [https://www.cbsnews.com/news/toyota-unintended-acceleration-has-killed-89/] 


= GM brake issues — most saved by human 
e 293 injuries, 2111 crashes 
@ 10,861 NHTSA complaints https://www.nytimes.com/1999/07/22/us/gm-admits-brake-flaws-after-inquiry.html 


= Will an ADS be as successful at fault mitigation as humans? 
e ADS will need to deal with heavy-tail issues © 2021 Philip Koopman 40 









Automotive Software Quality Issues 


RECALL CAMPAIGNS BY ELECTRONIC COMPONENTS & YEAR 
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@ SOFTWARE INTEGRATION 
@ SOFTWARE DEFECT 
IEC 


Source: Stout 2020 Automotive Defect & Recall Report 
IEC is integrated electronic components (hardware) 
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Automotive Software Has Defects Mellon 


University 





= Small sampling of NHTSA recalls (i.e., confirmed bugs) 


See: https://betterembsw.blogspot.com/p/ ricialn deadly-automotive-software.html 
21V-071 Vehicle unexpected pulls to one side during evasive maneuver 

20V-213 Remote smart park continued motion after failsafe activation 

19E-070 Anti-rollback software causes unexpected vehicle motion 

19V-539 Forward collision avoidance does not detect stationary vehicle 

19V-351 Regenerative braking failure reduces deceleration 

19V-075 Transmission unexpected downshift to first gear causes loss of control 
18V-621 Automatic braking cancelled / ABS locks up wheels 

18V-607 Active Lane Keeping Assist does not intervene in lane departure 
17V-713: Engine does not reduce power due to ESP software defect 


17V-686 and MANY others: Airbags disabled 
15V-460 and others: Airbags deploy when they should not 
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Example Required ADS Fault Handling tii. 


= Tire blowout/wheel detachment 
e ADS: perform controlled stop (or run-flat tire operations) 
= Service brake failure 
e ADS: downshift/regen braking, apply parking brake, runaway ramp 
= Catastrophic sensor failure 
e ADS: dead reckon to stop using most recent object trajectories 
= Uncommanded acceleration 
e ADS: de-energize engine/motors, apply forceful brakes 


= Main battery fire 
e ADS: shed electrical load, stop vehicle, passenger evacuation 
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Controllability Without A Human Driver Uaivorsity 


= What happens when there is no Bpomredsss [——aq a [| far 
human to exert controllability? 
e Own vehicle human driver? 
e Other vehicle human driver? 
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= Some combination of: 


e ADS will need to control faults to 
attain C1 or C2 


e Vehicle will have to upgrade 
subsystems to C3 (“uncontrollable”) m |SO 26262 Driver Controllability: 


nek by? e C1 = Simply controllable 
= Potential for significant ASIL «G7 = Normally cohtrollabie 


incr r whole vehicl rf. 
ein Sg aha ase y . ide e C3 = Difficult / uncontrollable 
e Many ADS control requirements ; 
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No Human Driver to Blame | Linettty 





= “Computers wont drive drunk’ .. but... 
e Drunk/DUI is only 28% of fatalities (US 2019) 


[https://crashstats.nhtsa.dot.gov/Api/Public/Publication/813060] 
e Automated Driving Systems (ADS) will 
likely make d/fferent mistakes 
— Perception/classification errors 
— Brittle in face of surprises (unknown unknowns) 


= What happens with ADS “driver error’? 
e Every AV crash is a product liability lawsuit 
waiting to happen 
e Eventually, no human driver to absorb blame 
— What about Driver monitor system (DMS) failures? 
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Drivers do more than just drive a ics bo 
e Occupant behavior, passenger safety é 
e Detecting and managing equipment faults 


Operational limitations & situations 
e System exits Operational Design Domain ntps://bitly/26vDKUN 
e Vehicle fire or catastrophic failure 

e Post-crash response 





Interacting with non-drivers ‘ila ie 
e Pedestrians, passengers a °F ae 
e Police,emergency responders al 
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Handling updates 


e Fully recertify after 
every weekly update? 


e Security in general 





Vehicle maintenance 
e Pre-flight checks, cleaning 
e Corrective maintenance 


Supply chain issues 
e Quality fade 


https://bit.ly/2VavsjM 


e Supply chain faults Is windshield cleaning fluid life critical? 
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Changing Role of 
Human Driver 


«* No human driver to blame for crashes 
“* ADS handles vehicle equipment failures 


«* ADS handles non-ADS software failures 
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